A Neural Autoregressive Framework for Collaborative Filtering

Restricted Boltzmann Machine (RBM) is a two layer undirected graph model that capable to represent complex distributions. Recent research has shown RBM-based approach has comparable performance with, even performs better than previous models on many collaborative filtering (CF) tasks. However, the intractable inference makes the training of RBM sophisticated, which prevents it from practical application. The Neural Autoregressive Distribution Estimator (NADE) is inspired by RBM, but it provides tractable distribution using an autoregressive approach. We describe a novel neural framework for collaborative filtering called NACF, which is extended from NADE, providing comparable performance with previous models but an easier training procedure than RBM. We apply the autoregressive approach of NADE to CF tasks by extending it in many ways. First, we model the user-item ratings with different visible units, among which the linear output units perform best. We propose the dual reversed ordering approach to relieve the data sparsity and bias caused by single random ordering. Further, we show that it is easy to combine with other information to improve performance, which is promising for practical application. Finally, we show the NACF model can be easily trained compared to RBM-based models, and the itembased NACF yields better performance than corresponding RBM on two MovieLens datasets.